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Abstract 

This  paper  determines  the  rale  of  convergence  of  a  class  of  block  Jacobi  iterative  schemes  when 
the  schemes  are  applied  to  a  general  class  of  problems.  Among  these  iterative  schemes  are  the  9-line 
and  9-plane  block  Jacobi  schemes,  while  the  general  class  of  problems  include  discretizations  of 
elliptic  and  parabolic  equations  whose  coefficients  do  not  depend  on  one  of  the  spatial  variables. 


AMS  (MOS)  Subject  Classification:  65F10 


Key  Words:  Iterative  Methods,  Block  Jacobi  Schemes,  Block  Gauss-Seidel  Schemes,  Block  Succes¬ 
sive  Over-Relaxation  Schemes 


Work  Unit  Number  3  —  Numerical  Analysis  and  Scientific  Computing 


*  Southern  Methodist  University.  Mathematics  Department,  Dallas,  Texas  75275 
Sponsored  by  the  United  States  Army  under  Contract  No.  DAAG29-80-C-0041  and  the  De¬ 
partment  of  Energy  under  Contract  No.  DE1-AL02-82ER12046A000. 


SIGNIFICANCE  AND  EXPLANATION 


The  solution  of  many  elliptic  and  parabolic  partial  differential  equations  lead  to  the  need  to 
solve  large  linear  systems.  With  the  usual  serial  computer  architecture  point  iterative  schemes 
frequently  led  to  the  efficient  solution  of  many  of  these  systems.  In  a  point  iterative  scheme  the 
current  estimate  of  the  solution  is  improved  in  a  repetitive  fashion  by  modifying  only  one  component 
of  the  solution.  The  advent  of  vector  and  parallel  computer  architectures  now  allow  the  efficient 
solution  of  these  systems  by  using  block  iterative  schemes.  In  a  block  iterative  scheme  the  current 
estimate  of  the  solution  is  improved  in  a  repetitive  fashion  by  modifying  several  components  of  the 
solution.  Since  point  iterative  schemes  can  be  viewed  as  particularly  simple  block  iterative  schemes 
one  would  expect  to  find  that  block  iterative  schemes  can  potentially  converge  faster  than  point 
iterative  schemes.  In  this  paper  the  rate  of  convergence  of  one  class  of  block  iterative  schemes  is 
precisely  determined. 

/ 
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The  responsibility  for  the  wording  and  views  expressed  in  this  descriptive  summary  lies  with 
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THE  RATE  OF  CONVERGENCE  OF  A  CLASS  OF  BLOCK  JACOBI  SCHEMES 


Warren  E.  Ferguson.  Jr.* 


1 .  Introduction 

This  paper  determines  the  spectrum  of  a  class  of  block  Jacobi  iterative  schemes  when  these 
schemes  are  used  to  solve  linear  systems  of  the  form 

Ax  =  b  with  A  =  lrid(-7,T,  -1) 

a  square  matrix  of  block  order  n.  In  this  analysis  it  will  be  assumed  that  I  denotes  the  identity 
matrix  of  order  m  and  that  T  is  a  symmetric  positive  definite  matrix  of  order  m  whose  eigenvalues 
are  all  greater  than  two.  Such  problems  arise  quite  naturally  when  elliptic  and  parabolic  parital 
differential  equations,  whose  coefficients  do  not  depend  on  one  of  the  spatial  variables,  are  dis¬ 
cretized.  Of  course,  knowing  the  spectrum  of  an  iterative  scheme  allows  one  to  precisely  describe 
Its  rate  of  convergence  a  . 

Among  the  block  Jacobi  iterative  schemes  considered  are  the  ^-line  block  Jacobi  schemes  and 
their  higher  dimensional  variants.  These  schemes  have  been  described  and  analyzed  extensively 
by  others  il,2,3,4  .  As  an  application  of  this  analysis  the  spectral  radius  of  the  7-line  block  Jacobi 
scheme  is  determined  when  it  is  applied  to  the  two-dimensional  Dirichlet  version  of  Laplace’s 
equation  in  the  unit  square 

To  every  block  Jacobi  scheme  there  is  a  naturally  related  block  SOR  scheme! 5  .  Since  the 
block  Jacobi  matrices  considered  here  are  consistently  ordered  2-cyclic  matrices  then  the  results 
of  Varga  can  be  used  to  determine  the  optimal  relaxation  parameter  for  the  block  SOR  scheme. 

In  this  paper  block  matrices  will  be  used  extensively  These  block  matrices  all  have  the 
property  that  the  blocks  on  the  diagonal  are  themselves  square  matrices.  Furthermore  the  notation 
diagO  and  trid()  will  be  used  to  denote  diagonal  and  tridiagonal  matrices  respectively 

2.  The  Block  Jacobi  Scheme 

The  block  Jacobi  scheme  to  be  considered  is  but  one  member  of  a  general  class  of  iterative 
schemes  which  are  frequently  used  to  solve  the  linear  system 

At  -  b  (2  la) 

This  general  class  of  iterative  schemes  has  the  form 

Bz'*'’ =  -  1)  for  i-  1.2.3.  (2  Ih 
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where  is  an  initial  approximation  to  x  while  B  and  C  describe  a  splitting  of  Aj5;;  i.e.  B  is 
nonsingular  and 

A  =  B-C.  (2.1c) 

It  is  well-known  that  the  rate  of  convergence  of  the  iterative  scheme  (2  1b)  is  described  by  the  root 
A  =  />  of 

XB~  C  0  (2. Id) 

having  largest  magnitudei5:.  In  equation  (2. Id)  the  function  •  represents  the  determinant  function 
Clearly  (2. Id)  is  a  generalized  eigenvalue  problem. 

In  this  analysis  it  will  be  assumed  that 


>4  =  trid(-/.r.-/)  (2.2a) 

is  a  square  matrix  of  block  order  n  where  the  identity  matrix  I  and  T  are  square  matrices  of  order 
m.  The  matrix  T  is  also  assumed  to  be  a  symmetric  positive  definite  matrix  whose  eigenvalues  are 
all  greater  than  two.  Since  T  is  a  symmetric  matrix  then  there  is  an  orthogonal  matrix  P  which 
diagonalizes  T;  i.e. 

PTP  =  diag(t,,t2,...  ,t„)  (2.26) 

where  P'  denotes  the  transpose  of  P.  Without  loss  of  generality  it  will  be  supposed  that  the 
eigenvalues  ,<2 _ of  T  are  ordered  so  that 

2<ti<ti<  ...  <t„,.  (2.2c) 


Consider  the  case  n  =  pq  where  p  >  2  and  q  >  1  are  integers.  The  block  Jacobi  scheme  to  be 
considered  is  of  the  form  (2.1b)  with 

B  =  trid(0,A/,0)  and  C  =  trid(.V,0.  A") .  (2.3a) 

both  square  matrices  of  block  order  p  where 


M  =  trid(-;,T,-/)  and 


■0  ..  0  r 

0  ...  0  0 

.0  ...  0  0. 


(2.36) 


are  themselves  square  matrices  of  block  order  q.  Since  B  is  a  symmetric  positive  definite  matrix 
it  follows  that  the  roots  of  (2. Id)  are  real.  It  will  be  assumed  that  g  >  2  since  the  roots  of  (2. Id) 
when  g  =  1  have  already  been  determined  by  Arms.  Gates  and  Zondek  1  . 


3.  Simplification  of  the  Generalized  Eigenvalue  Problem 


One  of  the  basic  tools  used  to  determine  the  roots  of  (2. Id)  is  the  following  explicit  represen¬ 
tation 


M-i  _  /  5-‘(7’)5,_,(r)S,_,(r),  when  .  < 

-  I  5->(7’)5,-,(r)S,_.(r).  when  >  j 


(3.1a) 


-2- 


of  A/  *  •  (Af,Y)  Bank  6  .  Here  Si(t)  denotes  the  degree  Chebyshev  polynomial  of  the 

first  kind  scaled  for  the  interval  0,2:.  Since  the  eigenvalues  of  T  are  greater  than  two  then 


=  ~sinh~^)~'  cosh(.^)-^  (3.16) 

IS  an  appropriate  representation  of  these  polynomials.  Based  on  this  representation  of  Af“*  it 
follows  that  (2. Id)  can  be  reduced  to  the  equation 

'.XI  -  D  =0  (3.1c) 

where 

D  =  fl-‘C  =  Trid(A/-*A’.0,A^-’A"). 

M-^N  =  i0,...,0.Af:,*.,and  (3. Id) 

M-‘A"  =  iA/-‘.0,...,0^ . 

Here  A/~j  and  denote  respectively  the  first  and  last  block  columns  of  A/“^. 

The  further  simplification  of  equation  (2.4c)  is  best  explained  by  considering  the  special  case 
p  =  3.  In  this  case  XI  -  D  has  the  following  nonzero  structure 


(1)  (2)  (3)  (3)  (2)  (3)  (3)  (2)  (1) 


(xi 

-F 

XI 

XI 

-G 

-G 

XI 

-F 

XI 

-F 

XI 

-G 

-G 

XI 

XI 

-F 

XI  J 

(3.2a) 


where 

F  =  S-\T)  and  G  =  5,-»(7’)5,-,(r)  (3.26) 

By  expanding  the  determinant  of  the  matrix  described  by  (3.2a)  columnwise  along  the  block 
columns  labeled  (1)  and  (2),  then  interchanging  the  adjacent  block  columns  labeled  by  (3),  it 
follows  in  the  general  case  that  (3.1c)  simplifies  to 

|£.j  ^  Q  (3  2c) 


-3- 


The  assumption  (3.3i)  is  justified  in  view  of  equations  (2.2c).  (3.1b)  and  (3.3b). 


4.  Solution  of  the  Generic  Equation 


The  first  step  in  the  determination  of  the  roots  of  the  generic  equation  (3.3g)  is  to  realize 
that  the  matrix  described  by  (3.3h)  is  in  actuality  a  tridiagonal  matrix.  If  Hi,  denotes  the  matrix 
obtained  by  deleting  the  last  row  and  column  from  if/^  it  follows  that  the  determinants  of  Hk  and 
H)i  satisfy  the  recurrence  relations 


Hk-i  -  g  Hk  ,  and 

Hk  =  ~f^\Hk-i  -g  Hk-i  . 


(4.I0) 


Consequently  it  is  easily  shown  that 


'Hk;' 

[y*-A’  g/*  1 

k  1 

l' 

0 

(4.16) 


Since  \Hk\  depends  on  A*  it  follows  that  the  roots  of  (3.3g)  occur,  as  expected,  in  positive/negative 
pairs. 

The  next  step  in  the  determination  of  the  roots  of  the  generic  equation  (3.3g)  consists  of  the 
localization  of  its  roots.  Recall  that  one  implication  of  Gerschgorin’s  theorem  is  that  irreducibly 
diagonally  dominant  matrices  are  never  singular [5i.  By  applying  this  result  to  the  matrix  Hk  in 
(3.3h)  and  its  column  permuted  form 


trid  ^ 


-/  0 
0  0 


A  -g 
[-g  A 


0  0 
0  -/ 


(4.2a) 


it  follows  that  the  roots  of  (3.3g)  lie  either  in  the  open  subinterval  (ff  -  /,p  -  /)  of  the  positive 
real  line  or  its  negative  image  (-g  -  /,  -  j  +  /) 


Assume  that  A  lies  in  the  open  subinterval  {g  -  f,g  +  f)  of  the  positive  real  line.  For  these 
values  of  A  the  two  eigenvalues  and  m  of  the  matrix 


-y  -n 


are  distinct  and  admit  the  representation 


Pi  =  A/e’^  and 


Pj  =  A/c-" 


(4.3o) 


(4.36) 


where  9  is  that  angle  in  the  open  inverval  (O.ir)  defined  by  the  equation 

y*- A^-r 


cos {9)  = 


2A/ 


Note  that  9  increases  from  0  to  x  as  A  increases  from  g  ~  f  to  g  / .  The  matrix 

Pi  + /*  Pj-^/* 

~9  -g 


(4.3c) 


(4.3d) 


-5- 


diagonalizes  the  matrix  in  (4  3a)  and  so  a  short  calculation  shows  that 


'\ffk\' 

1 

(p,.  />*-  (P2-/')PJ' 

.‘Mkl. 

Pi  -  P2 

-fflPl-Pi) 

From  the  representation 

Si{2cos(6))  -  ' ■■  0  <  6  <  -K 


of  the  Chebyshev  polynomials  of  the  first  kind  (4  3e)  can  also  be  written  as 


iHfc: 

'Hfc, 


-(A/) 


k-\ 


A/ 

-gSki 


ps,., 

2  _  >2  _  ;2 


X‘-P 


A/ 


(4.3e) 


(4.3/) 


(4Sg) 


It  follows  that  (4.3g)  holds  for  all  A  since  both  sides  are  polynomials  in  A  which  agree  in  value  for 
all  A  in  the  nonempty  open  interval  (g  -  f.g  ~  /). 


It  is  now  a  simple  matter  to  describe  the  roots  of  the  generic  equation  (3.3g).  The  roots  of 
(3.3g)  occur  in  positive/negative  pairs  with  the  positive  roots  admitting  the  representation 


A  -  -  /  cos  (fi)  »  y'S*  “  /*  (P)  (4,4a) 

where  0  is  any  one  of  the  k  distinct  roots  of 

tan  (Ik  -  IW)  -  (4,46) 

f^sin^  (0) 

in  the  open  interval  (0.  tr).  As  illustrated  by  Figure  1  when  /r  =  3  and  g  =  1,1/.  there  is  exactly 
one  root  of  (4,4b)  in  each  of  the  disjoint  intervals 

. ‘ 

Noip  that  as  Ir  -  00  the  intervals  Ji,J2 . Jk  densely  fill  the  interval  (O.tt)  with  the  points  in 

Jk  uniformly  approaching  n  Consequently  the  sharpest  upper  bound  for  the  largest  root  of  (3.3g) 
that  IS  valid  for  all  k  is 


X  —  g  ^  f  (4.4d) 

Note  that  this  is  the  same  upper  bound  on  the  roots  of  (3,3g)  described  by  Gerschgorin's  theorem, 

h  The  Spectrum  of  the  Block  Jacobi  Scheme 

Using  the  results  of  the  previous  section  the  specturm  of  the  block  Jacobi  scheme  can  be 
described  a.s  follow^  \s  shown  by  (3,2c).  exactly  n  2(p  1)  m  roots  of  (2. Id)  are  zero.  Of  the 

remaining  2(p  I  nonzero  roots  of  (2  Id)  half  of  the  roots  are  positive  and  the  other  half  are  the 
negative  images  of  'he  pipsitive  roots  The  (p  l)m  positive  roots  can  be  divided  into  m  clusters 


each  containing  p  -  1  roots.  The  such  cluster  ronsi.sts  of  the  roots  of  (2. Id)  lying  in  the  open 
interval  (j/  -  fi,gi  +  fi)  and  can  be  obtained  from  (4.4a)  by  solving  (4.4b)  with  k  —  p  ~  I,  f  =  fi 
and  g  =  gt  for  the  p  -  1  roots  9  lying  in  (O.ir). 

The  clusters  described  above  are  also  the  same  clusters  predicted  and  observed  by  Kratzer. 
Parter,  and  Steuerwalt  4  .  .Note  that  the  radius  fi  of  the  /'**  cluster  decreases  rapidly  as^  increases. 

Since  /(  and  jj/  are  decreasing  functions  of  /  it  follows  that  the  root  p  of  (2. Id)  having  largest 
magnitude  belongs  to  the  cluster  of  roots  lying  in  (pi  -  fi  g\  ~  /j)  It  can  be  obtained  from  (4.4a) 
by  solving  (4.4b)  with  k  =  p  -  \.  f  =  }i  and  g  =  g\  for  the  largest  root  9  lying  in  (0,  tr).  Of  course, 
knowledge  of  p  allows  one  to  determine  the  optimal  relaxation  parameter  w  for  the  block  SOR 
methods  naturally  related  to  this  block  Jacobi  scheme!5  .  Since  the  iteration  matrix  for  this  block 
Jacobi  scheme  is  a  consistently  ordered  2-cyclic  block  matrix  it  follows  that 


_ ^ _ 

1  -  -  P- 


(5.1a) 


is  the  optimal  relaxation  parameter. 

As  stated  in  the  previous  section,  the  sharpest  upper  bound  on  p  which  is  valid  for  all  p  is  the 
bound 


When 


p  ~  /i  -t-  Pi  . 


(5.2a) 


1 


T  =  trid(-l,4,-l) 

is  a  square  matrix  of  order  m  then  it  is  well-known  that 

<1  =  2 4  sin*  (-?rh)  where  h  = 

2  m  -t-  1 

After  a  straightforward  Taylor  series  expansion  it  follows  that 

P  ~  1 


(5,2b) 

(5.2c) 

(5.2c) 


a  well-known  result  of  Parter(2'. 
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